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SUMMARY 

A  great  deal  of  effort  has  been  expended  in  connec¬ 
tion  with  the  use  of  digital  computers  to  play  chess  and 
checkers.  The  most  successful  has  been  the  checker— playing 
program  of  Samuel.  It  is  of  some  interest  then  to 
indicate  how  the  theory  of  dynamic  programming  can  be 
used  to  determine  optimal  play  in  the  great  majority  of 
Pawn-King  end-games  in  Chess,  with  computers  currently 
available,  and  in  all  probability,  to  determine  optimal 
play  for  the  entire  game  of  checkers.  Here  we  shall 
outline  the  basic  methods  which  are  of  interest  in 
themselves,  involving  as  they  do  the  concept  of  semigroups 
in  structure. 

Any  views  expressed  in  this  paper  are  those  of  the 
author.  They  should  not  be  interpreted  as  reflecting  the 
views  of  The  RAND  Corporation  or  the  official  opinion  or 
policy  of  any  of  its  governmental  or  private  research 
sponsors.  Papers  are  reproduced  by  The  RAND  Corporation 
as  a  courtesy  to  members  of  its  staff. 

This  paper  will  be  submitted  to  the  Proceedings  ^f 
the  National  Academy  of  Sciences  for  publication. 
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1,  Introduction.  A  great  deal  of  effort  has  been 

expended  in  connection  with  the  use  of  digital  computers 

to  play  chess  and  checkers.  The  most  successful  has  been 

the  checker-playing  progra-n  of  Samuel^.  It  is  of  some 

interest  then  to  indicate  how  the  theory  of  dynamic 
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programming  *  can  be  used  to  determine  optimal  play  in 
the  great  majority  of  Fawn— King  end-games  in  Ch^ss,  with 
computers  currently  available,  and  in  all  probability,  to 
determine  optimal  play  for  the  entire  game  of  checkers. 
Here  we  shall  outline  the  basic  methods  which  are  of 
interest  in  themselves,  involving  as  they  do  the  concept 
of  semigroups  in  structure.  Essential  to  our  approach 
are  the  ideas  of  recurrence  and  transitivity  which  liave 
proved  of  such  value  in  topological  dynamics  and  in  the 
theory  jf  Markov  chains.  Detailed  results  and  numerical 
examples  will  be  presented  in  subsequent  papers. 

2.  Chess  as  a  Mul^is^age  Dec  is  im^  Process ,  Let  p 
denote  the  state  of  the  game  in  Chess,  with  White  to  move. 
The  specification  of  p  involves  the  listing  of  the 
positions  of  all  of  the  pieces  and  pawns,  as  well  as 
information  concerning  castling  and  pawns  "en  passant." 
Introduce  the  function 


f(p)  -  1 

if 

P 

is 

a 

win  for  Wliite, 

-  -  1 

.f 

P 

is 

a 

loss  for  W^iite, 

-  0 

if 

P 

is 

a 

draw. 

Let  the  effect  of  a  move  hv  Wliite  be  denoted  bv  T,,, 

^  W 

let  Tg  denote  the  result  of  a  move  by  Black.  The 


and 
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consequence  of  a  move  by  White  followed  by  a  move  by  Black 
is  to  replace  p  by  TgT^,  The  principle  of  optimality 
then  yields  the  basic  functional  equation 

f(p)  ■  max  min  f(TgT^),  (2.2) 

which  in  principle  determines  both  the  function  f(p) 
and  optimal  play,  under  appropriate  boundary  conditions. 

As  it  scands,  however,  the  equation  is  useless  computa¬ 
tionally  because  of  the  dimensionality  barrier,  and  it  is 
useless  analytically  because  of  our  lack  of  knowledge  of 
the  intrinsic  structure  of  chess.  It  is  this  ignorance 
which  prevents  the  application  of  approximation  techniques 
to  the  study  of  (2.2). 

3.  Recurrence— transitivity— stratification.  To  make  some 
progress,  we  lower  our  sights  and  concentrate  upon  the 
end-game  in  which  only  Pawns  and  Kings  are  on  the  board. 
Despite  the  fact  that  a  direct  application  of  (2.2)  is 
still  not  possible,  the  technique  of  structural  stratifi¬ 
cation  described  below  enables  us  to  introduce  a 
sequential  procedure  which  is  computationally  feasible 
with  available  computers. 

To  this  end,  we  distinguish  between  two  types  of 
moves,  those  which  are  recurrent  and  those  which  are 
tjran^sUiiye.  A  King— move,  not  involving  capture,  is 
recurrent  in  the  sense  that  it  is  reversible;  a  Pawn- 
move,  or  a  King— move  involving  capture,  is  transitive 
in  the  sense  that  it  is  irreversible.  Thus  the  second 
class  of  moves  partake  of  the  inexorable  quality  of  the 
arrow  of  time.  The  change  in  structure  cannot  be  undone. 

Associated  with  p,  in  the  end-game,  there  is  a 
set  Sq  containing  all  positions  derived  from  p  by 
means  of  recurrent  moves.  Each  position  p  thus  defines 
an  equivalence  class.  There  are  a  finite  number  of  moves 
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which  change  Che  structure  in  an  irreversible  fashion, 
to  wit,  the  pawn  moves  and  the  King  moves  involving 
capture.  Let  P2^*P2^  •  •  •  >Pjwj  denote  the  resultant  states 
and  ,  S2 ,  . .  .  ,  Sj^,  the  associated  sets  of  equivalent 
states  under  recurrent  moves.  Schematically: 


Now  we  introduce  a  stratification  technique,  a 
method  of  partitioning  the  original  state  space.  Let 
f(p)  be  the  function  defined  above,  and  introduce  the 
new  functions  fQ,  f  . . . ,  fj^  defined  in  the  following 
way: 

fo^P)  “  P  ^  Sq, 

(3.1) 

f^Cp)  “  f(p)i  p  e  s^. 

Since  the  functions  f.  are  defined  only  on  the  sets 

^  2 

S^,  we  see  that  not  more  than  (6^*)  possible  different 
states  p  can  arise  for  p  e  Consequently,  there  is 

no  trouble  in  storing  f^(p)  in  rapid— access  storage. 
Actually,  the  number  of  equivalent  states  is  generally 
considerably  smaller,  taking  account  of  restrictions  on 
the  location  of  Kings,  and  still  less  if  we  introduce 
some  rudimentary  ideas  of  strategy  and  tactics.  This 
last  point  is  important  computationally  if  there  are  many 
Pawns  on  the  board. 

Similarly,  starting  with  the  new  set  S^,  we 

introduce  sets  S..,  associated  functions  f..,  and  so 

tj  ij 

on.  Using  the  functional  equation  of  (2.2),  we  now 
obtain  recurrence  relations  connecting  cc  tiguous  functions, 
fQ  in  terms  of  the  f^,  f^  in  terms  of  the  f ,  and 


so  on. 
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To  start  the  calculations,  we  use  a  number  of 
boundary  conditions,  such  as  the  fact  that  the  game  is 
obviously  a  draw  when  no  pawns  remain  on  the  board,  the 
fact  that  we  can  recognize  a  win,  loss,  or  draw  when  only 
one  pawn  remains  on  the  board,  and  similarly  that  we  can 
recognize  a  win,  loss,  or  draw  as  soon  as  a  pawn  promotes. 
In  some  cases,  there  may  be  some  doubt  concerning  this 
last  statement,  which  is  why  we  claim  only  "the  great 
majority  of  cases."  In  actual  game  play,  there  should  be 
no  difficulty.  It  is  easy,  however,  to  conceive  of  problem 
positions  where  there  could  be  a  question, 

4.  Discussion.  Even  with  the  techniques  described  above, 
we  cannot  handle  King— Piece— Pawn  endings  with  chc  computers 
currently  available.  It  seems  reasonable  to  predict, 
however,  that  these  techniques  will  be  powerful  enough 
with  the  computers  available  within  ten  years  or  so.  The 
middle  game  remains  as  far  away  as  ever  as  far  as  any 
rigorous  treatment  is  concerned. 

5,  Checkers .  The  initial  stratification  in  the  game  of 
checkers  is  most  easily  done  by  keeping  track  of  the 
number  of  men  that  remain  on  the  board,  and  structure,  if 
necessary.  In  the  beginning,  all  moves  are  necessarily 
transitive  moves.  Involving  changes  in  structure  and 
capture.  As  soon  as  a  King  is  achieved  by  either  side, 
we  have  the  possibility  of  recurrent  moves.  The  boundary 
condition  we  impose  is  that  we  can  recognize  whether  a 
game  is  a  win,  loss,  or  draw  as  soon  as  five  or  more 
Kings  are  on  the  board.  This  is  not  the  case,  certainly 
as  far  as  problem  positions  are  concerned,  but  does  seem 
to  be  the  case  as  far  as  actual  play  is  concerned.  With 
bigger  computers,  this  restriction  can  be  removed  and  it 
seems  safe  to  predict  that  within  ten  years,  checkers 
will  be  a  completely  decidable  game. 
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